AD-Suite - A Test Suite for Algorithmic Differentiation

Algorithmic differentiation (AD) is a widely accepted methodology to obtain derivatives of scientific code for use in optimization, integration methods, and sensitivity analysis. Currently about 60 AD tools are listed on autodiff. org to compute sensitivity information. Several years have been invested in the research and development of popular implementations such as ADOL-C, CppAD, OpenAD, and Tapenade. In most cases, however, the test code that comes packaged along with the tools comprises either toy or academic examples that are far removed from real-life applications. In general testing scientific software is difficult. A common reason is the lack of separation between theory and code in a scientist’s mind that limits testing to a mere verification of theory [7]. The lack of good test codes for AD in particular can be attributed to a couple of factors. One is that tools are written in and for different programming languages — C/C++, Python, Java and Fortran. The second is that AD is usually implemented by using operator overloading or source transformation and requires special formatting of the input code. These two factors contribute to the need for changes in the original simulation code or even recoding of the whole program in order to apply the developed AD software and get appropriate results. Drawing inspiration from projects that attempt to solve the problem of testing scientific software in other fields such as [1], [3], [4], [5], and [6], we propose to create AD-Suite: a test suite for AD and a classification that describes the applications included in that suite. To the best of our knowledge, there exists neither a classification of applications nor a test suite for AD. Although, projects such as SifDEC provide the ability to generate input for AD tools, they deal mostly with nonlinear optimization and hence narrow the scope for an AD classification suite; see, for example, [2], where most or all of the problems within the CUTEr test set have sparse derivatives. An additional barrier is that one needs to create a Sif-to-any-programming-language source transformation tool in order to generate test cases in different languages and the required special format (and this of course has to be tested independently). Also, most users are understandably unwilling to learn a new format and recode their application in order to apply AD. Instead, we aim to crowd-source codes in multiple languages, spanning several application areas and having code structure that is typical in practice. In particular, we hope that AD users will actively contribute to the test suite with minimal additional effort, because further advancement of open-source AD tools is highly dependent on the variety and richness of the provided applications and examples. The examples also yield instructive templates for other interested users who want to apply AD. For AD developers, AD-Suite will be useful in a wide range of scenarios. For example, available applications will enable AD developers to better understand the needs of the users, anticipate future research areas; and design appropriate drivers. Moreover, it can be used to validate the correctness of a new implementation and compare the results. It also allows for performance comparison with other tools and different approaches on a “suitable” set of problems. We expect AD-Suite to be publicly available on autodiff.org and contain for each problem all necessary information. As part of this effort, we envision a webform where we will collect the information along with each submission in a standard format. Through this endeavor, we hope to strengthen the AD community by bringing together AD developers, industrial partners, and users of AD.